Final Report – CS 6604 Spring 2017
نویسندگان
چکیده
............................................................................................................................................ I TABLE OF TABLES ................................................................................................................................. III TABLE OF FIGURES .............................................................................................................................. IV 1 OVERVIEW .................................................................................................................................... 5 1.1 MANAGEMENT ................................................................................................................................. 5 1.2 CHALLENGES .................................................................................................................................... 5 1.3 SOLUTION DEVELOPED ....................................................................................................................... 6 2 LITERATURE REVIEW ..................................................................................................................... 7 3 REQUIREMENTS ............................................................................................................................ 8 4 DESIGN ....................................................................................................................................... 10 4.1 EVENTS CRAWLING DESIGN .............................................................................................................. 11 4.2 HBASE SCHEMA DESIGN .................................................................................................................. 14 5 IMPLEMENTATION ...................................................................................................................... 15 5.1 OVERVIEW ..................................................................................................................................... 15 5.2 TIMELINE ....................................................................................................................................... 15 5.3 TOOLS ........................................................................................................................................... 17 5.3.1 ArchiveSpark ....................................................................................................................... 17 5.3.2 D3.js .................................................................................................................................... 17 6 USER MANUAL ............................................................................................................................ 19 7 DEVELOPER MANUAL .................................................................................................................. 22 7.1 INTERNET ARCHIVE TOOL ................................................................................................................. 22 7.2 TUTORIALS FOR DEPLOYING EFC ....................................................................................................... 23 7.2.1 Install Dependencies ........................................................................................................... 23 7.2.2 Run EFC ............................................................................................................................... 23 7.3 TUTORIALS FOR DEPLOYING ARCHIVESPARK IN JUPYTER ......................................................................... 24 7.3.1 Install JDK 8 ........................................................................................................................ 24 7.3.2 Install Python 3.5 and Pip ................................................................................................... 24 7.3.3 Install Jupyter ..................................................................................................................... 25 7.3.4 Install Spark 2.1.0 ............................................................................................................... 25 7.3.5 Install ArchiveSpark ............................................................................................................ 26 7.3.6 Replace the Original Scala .................................................................................................. 26 7.4 TUTORIALS FOR DEPLOYING ARCHIVESPARK IN INTELLIJ ......................................................................... 27 7.4.1 Install Spark and Scala ........................................................................................................ 27 7.4.2 Deploy ArchiveSpark ........................................................................................................... 27
منابع مشابه
CS 3110 Spring 2017 Lecture 25 : Course Review and Final Exam Coverage
Lecture 3: More reduction rules, the notion of Currying and Uncurrying, and typing rules for some of the constructs. A key signature idea of the ML family of languages is introduced, the polymorphic types which in this version of the course are written with both the standard OCaml syntax ‘a, ‘b, ‘c, ... and with Greek letters as in the original articles on ML, e.g. α, β, γ, .... It would be goo...
متن کاملUnusual presentation of a patient with hemoglobin Constant Spring and immune hemolytic anemia
Abstract Introduction: Hemoglobin Constant Spring (Hb CS), a abnormal Hb characterized by elongated α-globin chain resulting from mutations of the termination codon in the α2 - globin gene , is the most common nondelitional α-thalassemic mutation and is an important cause of HbH like disease in Southeast Asia. Case Report: A 9- years-old female with immune hemolytic anemia and splenomegally...
متن کاملCs 6604: Data Mining
In the last lecture we discussed the relationships between different modeling paradigms such as the Bayesian approach, Maximum A Posteriori (MAP) approach, Maximum Likelihood (ML) approach, and the Leastsquares (LS) method. In this lecture we first prove that equivalence of LS and ML under the assumption of normally distributed error. Then, the notions of the naive Bayesian classifier and the L...
متن کامل